Discussion of "Spectral Dimensionality Reduction via Maximum Entropy"

Since the introduction of LLE (Roweis and Saul, 2000) and Isomap (Tenenbaum et al., 2000), a large number of non-linear dimensionality reduction techniques (manifold learners) have been proposed. Many of these non-linear techniques can be viewed as instantiations of Kernel PCA; they employ a cleverly designed kernel matrix that preserves local data structure in the “feature space” (Bengio et al., 2004). The kernel matrices of the first manifold learners were handcrafted: for instance, LLE uses an inverse squared graph Laplacian of the reconstruction weight matrix as kernel matrix, Isomap uses a centered geodesic distance matrix, and Laplacian Eigenmaps uses an inverse neighborhood graph Laplacian (Belkin and Niyogi, 2002).